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Summary 


Students who successfully transition to ninth grade are more likely to graduate from high 
school. In 2018, the School District of Philadelphia (SDP) developed an indicator for being “on 
track” at the end of a high school student’s freshman year. As part of the district’s overall strategy 
for improving graduation rates, school administrators currently use the Ninth Grade On-Track 
indicator as an early marker of progress towards graduation and to identify students in need of 
intervention. 


Using four cohorts of SDP data prior to its development and implementation, PERC found 
that one-third of first time freshman in 2013-2017 would have been flagged by On-Track 
indicator as “off-track” after their first year of high school.1 Although the On-Track indicator 
was not yet in place, the reports serve as a benchmark against which progress can be measured. 


Building upon this prior work, this report examines two of those same cohorts of off-track 
freshman in the expected graduating classes of 2017 and 2018, following them over time to 
understand which students were able to graduate and when they got back on track. Similar to 
our prior work, these findings describe student experiences that pre-date the establishment of 
SDP’s On-Track indicator. Thus, this report serves as baseline for the district to assess progress 
toward efforts to improve graduation rates through schools’ use of the On-Track indicator to 
identify and connect students to supports. 


We also assess graduation rates for students at varying degrees of being off-track and for 
subgroups of students and schools to examine compounding risk factors for off-track 
students. We examine graduation rates for students at varying degress of being off-track, i.e., what 
it means to be securely or marginally on track versus marginally, moderately, or very off-track at 
the end of 9th grade. The subgroup analyses draw attention to disparities in recovery from 9 grade 
off-track status and include student subgroups defined by gender, race/ethnicity, low-income 
status, 9th grade special education status and English Learner status, and 9‘ grade attendence and 
suspensions. We also examine the graduation rates for off-track students separately for those 
attending schools with small and larger enrollments and across school admission types. 


Finally, we examine when students get back on track and graduation rates of students who 
leverage credit recovery to get back on track. These findings apply to subsamples of students for 
whom data can support the analyses, thus are exploratory and require additional research to 
understand more fully. 


1 Crofton, M., & Neild, R. C. (2018). Getting On Track to Graduation: Ninth Graders’ Credit Accumulation in the School District of 
Philadelphia, 2014-2016. The Philadelphia Education Research Consortium.; Pileggi, M. & Strouf, K. (2018). On Track Across Four 
Cohorts: Ninth Grader On Track Patterns in the School District of Philadelphia, 2013-2017. The Philadelphia Education Research 
Consortium. https://www.phledresearch.org/on-track-4-cohorts 


The goal of examining these patterns is to understand which off-track students and which 
schools might need the most support. This report offers a first-time look at graduation rates for 
off-track students in SDP. We anticipate the findings will be informative to school and district 
administrators working to support high school students who fall off-track in ninth grade. These 
findings raise questions about whether some students and schools may benefit from more 
resources and/or different systems or approaches than others. The findings also point to the need 
for more research to examine why some students in some schools recover at higher rates than 
others and build evidence for effective interventions for getting students back on track. 


Key Findings 


e Compared to 82% of on-track ninth graders, only 57% of off-track ninth graders 
graduated on time—a difference of about 25 percentage points. This is a considerable, 
statistically significant difference and reinforces the utility of the On-Track indicator for 
flagging students as at risk of failing to graduate. 


e Among off-track 9t graders who were able to get back on track to graduate, two- 
thirds did so in 12th grade, possibly reflecting historical district priorities in place at the 
time these data were collected to target efforts to get seniors to graduate. A third of 
students got back on track during earlier grades. Of course, all students who got back on 
track in 12‘ grade graduated. Among those who got back on track before 12 grade, 85% 
stayed on track and graduated on time, suggesting that early intervention is valuable, 
though more research is needed to unpack intervention timing. An additional area of 
exploaration is the impact of the development and implementation of the district’s On- 
Track indicator in 2018 for motivating earlier intervention. 


e Off-track males had a graduation rate of 51%, nearly 12 percentage points lower than 
the graduation rate of 62% for off-track females. Males were also more likely to be off- 
track than their female peers.? The double disadvantage of males is troubling and highlights 
that substantial work remains to be done to improve their educational experiences. 


e Very poor attendence and suspensions are additional warning signals for off-track 
students. The graduation rate of off-track students receiving one or more suspensions in 9th 
grade was almost 10 percentage points lower than their off-track peers with no 
suspensions. Very poor attendance was an even stronger predictor of graduation. The 
graduation rate of off-track students with very poor attendance (average daily attendance 
below 80%) was 27 percentage points lower than their off-track peers with strong 
attendance (90% or higher average daily attendance). 


e ©Off-track students enrolled in Citywide and Special Admission high schools had 
substantially higher graduation rates than their off-track peers enrolled in 
Neighborhood high schools. The graduation rates at Neighborhood schools trailed those 
at other schools by more than 20 percentage points. 


e The farther off track a student was at the end of 9th grade, the lower their graduation 
rate. The graduation rate for students missing just one of the five indicators of being on- 
track was about 69%, compared to rates of about 50% of students missing 2 or 3 
requirements and about 27% for students missing 4 or 5. 


2 Pileggi, M. & Strouf, K. (2018). On Track Across Four Cohorts: Ninth Grader On Track Patterns in the School District of Philadelphia, 
2013-2017. The Philadelphia Education Research Consortium. https://www.phledresearch.org/on-track-4-cohorts 
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Only 20% of off-track 9t grade students in the Class of 2018 enrolled in credit 
recovery in 2017-18, though those who did had a much higher graduation rate than 
those who did not. Students who recovered credits during their senior year had a 
graduation rate of 84%, nearly 30 percentage points higher than their peers who did not 
attempt credit recovery in 2017-18. Students who attempted to regain credits, even if they 
were not successful, had a graduation rate of almost 12 percentage points higher than 
students who did not use credit recovery that year. 


Implications for policy and practice 


The findings in this report reinforce SDP’s recent adoption and use of the Ninth Grade 
On-Track indicator as a strategy for identifying and supporting off-track students to 
improve graduation rates. Of course, because the data presented here are correlational, 
we are cautious to not make strong claims that graduation rates would improve if recovery 
rates increased among off-track students. We encourage the district to build upon this body 
of research by examining the underlying reasons why students fall off track in 9th grade and 
evaluate efforts to help these students recover. 


While our report does not address what kinds of supports work for which off-track 
students, it suggests the need for targeted supports that attend to the reasons for 
increased risk of failing to recover for certain students. Importantly, findings suggest 
that attendence and suspension rates are additional warning signs that could identify off- 
track students for more targeted supports. Additionally, off-track male students were at a 
moderately greater risk of failing to graduate than their female peers. These disparities in 
rates of recovery call into question a one-size-fits-all approach to helping students get back 
and stay on track. 


The district should consider supporting principals, teachers, and other school 
support team members in understanding the implications of the degree to which 
students are off-track at the end of 9t» grade and adjust tracking systems to expand 
the indicator to include “marginal,” “moderate,” and “far off-track” categories. Our 
study does not examine school efforts to get students back on track nor does it document 
school-based approaches to supporting off-track students. However, it is likely that the 
causes of course failure for marginally off-track student are different than those for 
students who are far off-track. Thus, a one-size-fits-all approach will have muted effects, 
and schools armed with information about student’s degree of off-track may be able to 
tailored supports that address variable root causes. 


Helping students recover from being off-track will likely require different solutions 
in different school settings. Our finding that off-track students in Neighborhood schools 
have a graduation rate that is substantially lower than their peers in Special Admit or 
Citywide schools deserves more attention. Though our report does not address reasons for 
differences, it could be that Neighborhood schools, many of which have large proprotions of 
off-track 9 graders, might need additional capacity or different systems than schools with 
smaller off-track cohorts to help their students recover. 
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Why this study 


The successful transition to ninth grade is a critical predictor of high school graduation. For 
most ninth graders, the first year of high school brings broader course and extracurricular choices 
as well as increased academic challenges and greater independence. Evidence from Philadelphia 
and other cities shows that students who struggle to manage the transition from middle to high 
school and fall off track in ninth grade have a much higher risk of not completing high school, even 
if they were high-achieving students in middle school.3 


The School District of Philadelphia (SDP) defined “on track” at the end of the first year of 
high school as having earned at least one course credit in each core course (English, math, 
social studies, and science) and one additional course credit in any subject. SDP established 
this definiton in 2018 and demonstrated its utility for tracking student progress and motivating 
continuous improvement.‘ Importantly, SDP’s On-Track indicator takes into account which subject 
areas students earned credits in, not just the total number of credits they received, distinguishing 
the On-Track indicator from grade promotion requirements. The rationale for this approach is that 
graduation requirements include subject-specific credits, so being “on track to graduate” should as 
well.s 


With a definition in place, the “Ninth Grade Box 1: SDP’s Ninth Grade On-Track 
On-Track” indicator is currently being used to Definition 

provide principals, teachers, and other 

student support staff with an early indicator To be considered on track at the end of 
of student progress towards graduation. This ninth grade, a student must earn at least: 
simple, dichotomous indicator is used by schools * One course credit in each of the four 
to intervene with individual students to help core subjects (English, math, social 
them get back on track. studies, and science), and 

To help inform how to effectively support off- ol ai—Keloleliilelate| Rexel’ lex aeli—fellalakelayy 
track students, this study focuses on which subject. 


off-track students were able to graduate and 
when and how they got back on track. In 2018, 
PERC found that one-third of SDP high school students were off-track after their first year of high 
school, according to SDP’s new definition, and that off-track rates varied by student and school 
characteristics.6 Taking PERC’s earlier analyses further, this report investigates how off-track 
students progressed through their later high school years to understand which students from what 
kinds of schools were more likely to recover from being off-track and graduate on time. 


3 Allensworth, E. M. & Easton, J. Q. (2005). The On-Track Indicator as a Predictor of High School Graduation. Consortium on Chicago School 
Research. https://consortium.uchicago.edu/publications/track-indicator-predictor-high-school-graduation; Neild, R. C. & Balfanz, R. 
(2006). Unfulfilled Promise: The Dimensions and Characteristics of Philadelphia’s Dropout Crisis, 2000-2005. Philadelphia Youth 
Network. http://www.projectuturn.net/docs/Unfulfilled Promise Project_U-turn.pdf 

4 Wills, T. (2018). Defining 9" Grade Success: A New 9t Grade On Track Definition. The School District of Philadelphia. 
https://www.philasd.org/research/wp-content/uploads/sites/90/2018/05/On-Track-Focus-Brief_May-2018.pdf 

5 The School District of Philadelphia. (2018). Keeping Pace for Graduation: Suggested Credit Totals by Grade. 

https://www. philasd.org/research/wp-content/uploads/sites/90/2018/08/Suggested-Credit-Totals-by-Grade.pdf 

6 Crofton, M., & Neild, R. C. (2018). Getting On Track to Graduation: Ninth Graders’ Credit Accumulation in the School District of 
Philadelphia, 2014-2016. The Philadelphia Education Research Consortium.; Pileggi, M. & Strouf, K. (2018). On Track Across Four 
Cohorts: Ninth Grader On Track Patterns in the School District of Philadelphia, 2013-2017. The Philadelphia Education Research 
Consortium. https://www.phledresearch.org/on-track-4-cohorts 
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What the study examines 


This study examines the graduation rates of two cohorts of on- and off-track SDP ninth 
graders: first-time ninth graders in 2013-14 and 2014-15 (i.e., the expected graduating classes of 
2017 and 2018). This study includes students who completed their first year of high school at one 
of the 52 traditional high schools’ in SDP, excluding students at charter schools or alternative 
schools (such as evening schools within the SDP Opportunity Network). These analyses apply the 
SDP’s On-Track indicator to students who were in ninth grade prior to its establishment in 2018 to 
serve as a benchmark against which progress can be measured. 


In addition to overall graduation rates, we examine patterns by student and school 
characteristics to identify compounding risk factors for off-track students. We assess 
graduation rates for subgroups of off-track students, including by gender, race/ethncity, low- 
income status, 9th grade special education status and English Learner status, 9" grade attendence 
and suspensions, and school admission type and size. An analysis of these patterns is helpful for 
identifying disparities in the recovery from being off-track, the goal of which is to generate 
conversations about why disparities exist and ideas for providing supports that address them. 


We take a deeper look at the the degree to which students are off-track, examining cases right 
on the cusp of being on track (i.e, marginally off-track), those who are moderately off-track, and 
those who are far off-track. For marginally off-track students, who are missing only one credit, we 
also examine whether the consequences for graduation differ depending on the course subject that 
the student failed. 


Finally, this report examines when students tended to get back on track and how many of 
them leveraged credit recovery to do so. This analysis sheds light on how students have 
successfully gotten back on track to graduation in the past to inform strategies that could be helpful 
for off-track students in the future. These findings apply to subsamples of students for whom data 
can support the analyses, thus are exploratory and require additional research to understand more 
fully (see Limitations of this study section below). 


The research questions studied in this report are: 
e What percentage of off-track 9th graders were able to graduate on time? Within five years? 
e Ofthe students who were off-track in 9th grade but recovered to graduate on time, when 
did those students get back on track? 
e How do graduation rates of off-track students vary? 

o Bystudent characteristics (e.g. gender, race/ethnicity, low-income public assistance, 
and 9th grade special education status and English learner status, and 9t» grade 
attendance and suspensions)? 

o Byschool characteristics (i.e. admission type and size)? 

o By how many credits students were missing or by the type of credits that were 
incomplete? 


7 For the purposes of this report, we use the term “traditional” high school to refer to schools that held courses exclusively during the day 
and fit into one of the following SDP categories: Neighborhood, Citywide, or Special Admission high school. Schools in these categories may 
provide non-traditional programming, such as project-based learning, early college enrollment, work-based internships, or other innovative 
approaches, but for the sake of simplicity, in this report, they are all considered "traditional" high schools. Educational Options Programs 
(EOPs), entire schools serving students with alternative needs (e.g. Widener Memorial School), and other schools in SDP’s Opportunity 
Network are not included in our analyses. 
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e How do graduation rates of students who were marginally off-track compare to those who 
are marginally on track? 

e How many Class of 2018 students who were off-track at the end of their 9t grade year 
engaged in credit recovery during 2017-1878 Did engaging in Credit Recovery help these 
students graduate on time? 


Limitations of this study 


This study examines the association between being off-track and eventual graduation for students 
who were in school prior to the advent of the district’s 9t» Grade On-Track indicator. While we offer 
a descriptive analysis of patterns in graduation rates, the study has several limitations. First, this 
study is correlational and does not attend to the very-important questions of why students are off 
track, why some students are at increased risk of failing to recover, and whether and how different 
interventions might improve likelihood of graduation. Our results, however, 1) can be used as a 
benchmark against which district efforts can be compared, 2) suggest the need for On-Track 
indicators to be reviewed alongside other student-level data, and 3) draw attention to important 
disparities that should motivate the design and assessment of interventions. 


Second, our results may overestimate graduation rates because of sample restrictions. Students are 
excluded from the study if they permanently leave the district, because their graduation status 
cannot be determined. This may lead to overestimating graduation rates for off-track students 
because off-track students with greater risk of failing to graduate may be more likley to leave the 
district. This overestimation may be more pronounced the farther off track a student is, because 
far-off track students are more likely to leave the district (See Table B1 in Appendix). 


Several analyses in this report required additional sample restrictions. First, we restricted the 
analysis of when students got back on track to students who were continuously enrolled in the 
district, excluding those who, for example, transferred out of and then returned to the distirct. This 
sample restriction leaves only 43% of the full analytic sample available for this subanalysis, 
potentially overestimating graduation rates for these students to a greater degree than the main 
analysis. Second, due to data limitations, our analysis of graduation rates of students enrolled in 
credit recovery is restricted to the freshman Class of 2018 who enrolled in credit recovery in their 
4th year of high school (i.e., 30% of the sample of the full analytic sample and 62% of the off-track 
sample of Class of 2018 students). This is because data on whether a course was taken ina 
traditional format or an alternative credit recovery format was first captured in SDP’s data system 
for the 2017-18 school year. Students may have attempted or regained credits using alternative 
credit recovery formats before the 2017-18 year, but we could not identify those cases in the data. 
These analyses should be considered exploratory and built upon. 


8 This analysis was restricted to 2017-18 because that was the first year the data available clearly identified which courses were taken in a 
recovery format. 
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Data and Variables 


This study uses de-identified studentevel data for first-time ninth graders in the School District : 
of Philadelphia in 2013-14 and 2014-15 (expected graduating classes of 2017 and 2018). All : 
identifying information was removed before the research team received the data. 


The study sample was limited to students who finished their first year of high school at one of 
52 traditional School District of Philadelphia high schools (see Footnote 7 for definition of : 
“traditional high school”). : 


The analysis is restricted to students who did not permanently transfer out of SDP during their 


high school years, so that we could determine their graduation status. Our sample consisted : 
of 15,994 first-time 9!" grade students, 8,084 from the 2013-14 cohort and 7,910 from the 2014- : 
15 cohort. : 
Key variables are: : 


e 9th grade on-track to graduation status: Indicator for each student indicating if the 
student was on- or off-track to graduation at the end of their first 9! grade year. : 


e Degree of off-track: With five distinct requirements to being on track in 94 grade (see " 
Box 1), the research team developed an indicator for how far on or off track a 
student was based on how many of those requirements the student was missing: 
Securely on track, marginally on track, marginally off track, moderately off track, 


and far off track. : 
e On-time graduation status: Using the districts’s indicator for whether a student 
graduated, dropped out, or transferrred out of SDP (along with the year that : 


designation was reached), the research team created an indicator for “on-time” 
graduation, meaning that the student graduated within 4 years after entering 9th : 
grade. We also examined 5-year graduation status (results in Appendix C). : 


This study also includes analysis of student- and school-level characteristics and courses 
taken in 2017-18. See Appendix A for definitions of those variables. 
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Analytic Methods 


: Similar to high school graduation, we know from prior PERC research that on-track status at 
: the end of ninth grade is associated with school and student characteristics, in part because : 
: features of educational systems put some students and schools at a disadvantage (See : 


Footnote 6). 


The goal of this report is to isolate attention to disparities in recovery from being off track : 
apart from the risk of being off track, though these two processes are intertwined and jointly 
subject to the forces of inequality that are exerted on all students. 


To address this goal, we employ a multi-level logistic regression technique to estimate " 
marginal graduation rates for the study sample rather than presenting unadjusted rates. We 
: explain the difference in these two approaches here: 


e Unadjusted graduation rates can obscure the association between off-track status 
: and graduation, as both are influenced by the school a student attends and other 
systematic differences in educational experiences across student subgroups. : 
Differences between unadjusted graduation rates for on- and off -track 9 graders 
combine or confound factors associated with on-track status with those that are 


: associated with graduation. Factors that might influence both “on-track status” as : 
: well as graduation include student socio-demographic characteristics, ninth grade : 
: participation in Special Education and English Learner services, school attendance : 
: rates and behavior incidents in ninth grade, and the school the student attended in : 
ninth grade. : 
: e Marginal or “adjusted” graduation rates account for factors that are jointly : 
: associated with ninth grade on-track status and graduation (€.g., which school a : 
: student attends) in order to focus attention on the consequences of being off track : 
: for graduation. The isolation of this relationship from confounders helps decision : 
: makers narrow the scope of intervention points to address the key issue at hand: : 


how to support off-track students with varying risks of failing to graduate. However, 
this does not discount the importance of the social processes that put some students 
: disproportionately at risk of being off-track in 9th grade. As our recommendations 
suggest, attempts to support recovery for these students will necessitate a close : 
examination of root causes. 


. For the technical reader, Apppendix C presents the model building process and final : 
: regression model parameters. The preferred model was used to generate predicated : 
: marginal graduation rates, which were calculated with covariates set to the observed value : 
: of each student’s record and then averaged over the estimation sample. A comparison of : 
: unadjusted and marginal 4- and 5-year graduation rates, overall and by student and school : 


subgroups, can be found in Appendix C. 
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Graduation Rates for On- and Off-Track 9th Graders from the 
Classes of 2017 and 2018 


Compared to 82% of on-track ninth graders, only 56% of off-track ninth graders 
graduated on time, a difference of about 26 percentage points. 

This report examines graduation rates for two cohorts of 9‘ graders: the Classes of 2017 and 2018. 
First, we compare adjusted 4- and 5-year graduation rates for on- and off-track first-time ninth 
graders, using SDP’s definition of “on track,” i-e., students who, by the end of 9th grade, earned at 
least one credit in all four core courses and earned an additional credit in any subject area (Figure 
1).9 


Figure 1. Four- and five-year graduation rates for first-time ninth graders in 2013-14 and 2014-15 cohorts, adjusted 
for school and student characteristics, by 9°" Grade On Track status, N=15,994 


4 YEAR GRADUATION RATE 


82.2% 


56.4% 


5 YEAR GRADUATION RATE 


84.7% 


Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15 and who did not permanently transfer out of SDP during the study period. The 
statistical model adjusts for school attended, school type and size, student gender, race/ethnicity, cohort, and 9th grade 
measures of average daily attendance, number of suspensions, and low-income, English Learner, and Special Education 
status. The total sample size for this analysis was 15,994 students (10,646 on-track students and 5,348 off-track 
students). 


9 See comparison to unadjusted rates in Appendix C. 
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This figure shows: 


e The on-time graduation rate for off-track 9 grade students was considerably lower 
than that of their on-track peers. The on-time graduation rate for off-track ninth graders 
was 25.8 percentage points lower than that of their on-track peers (p<0.05). 

e Asimilar, but slightly smaller gap existed when looking at 5-year graduation rates. 
Off-track students had adjusted 5-year graduation rates that were 21.6 percentage points 
lower than their on-track peers (p<0.05). 


When Did Off-Track 9th Graders Get Back On Track? 


As school administrators assess how to best support off-track students, it is worth considering if 
the timing of when a student gets back on track is also important. Does it matter if the student gets 
back on track in the next year of high school? Or can they wait and make up those final credits in 
their senior year, when they might have more flexibility in choosing courses? 


Generally, students identified early can be matched with interventions to help them return more 
quickly to the on-time graduation track.1° While our study does not address whether early or later 
intervention is more effective (and for whom), the analysis below describes patterns for when off- 
track students were first able to get back on track towards graduation and if timing matters. As a 
reminder, this analysis represents student experiences that predate the development of the on- 
track indicator and its utilization in schools to identify students for early intervention. More recent 
cohorts may exhibit different patterns as a result of district efforts to intervene earlier. 


Nearly two in three students who were able to get back on track did so in 12th 
grade. 


Table 1 shows which year the off-track 9 graders first got back on track to graduation. The 
columns separate off-track 9 graders who graduated on-time (Column A) from those who did not 
(Column B) and also displays the unadjusted graduation rate of students who first got back on track 
during that grade. These findings represent a subset of off-track students with on-track status 
observable for all four years of high school (about 43% of all off-track students).11 We present 
unadjusted graduation rates because the sample size is too small to support a model-based 
approach. 


10 Heppen, J. B. & Therriault, S. B. (2008). Developing Early Warning Systems to Identify Potential High Schoo! Dropouts. 
https://files.eric.ed.gov/fulltext/ED521558. pdf; Jerald, C. D. (2006). Identifying Potential Dropouts: Key Lessons for Building an Early 
Warning Data System. Achieve, Inc. https://files.eric.ed.gov/fulltext/ED499838.pdf; Kennelly, L. & Monrad, M. (2007). Approaches to 
Dropout Prevention: Heeding Early Warning Signs With Appropriate Interventions. National High School Center, American Institutes for 
Research. https://files.eric.ed.gov/fulltext/ED499009. pdf; Neild, R. C., Balfanz, R., & Herzog, L. (2007). An early warning system. 
Educational Leadership, 65, 28-33.; Pinkus, L. (2008). Using Early-Warning Data to Improve Graduation Rates: Closing Cracks in the 
Education System. Alliance for Excellent Education. https://all4ed.org/wp-content/uploads/EWI.pdf 

11 A student might have missing data for other grades, thus are excluded from this analysis, if they: (1) left SDP district schools for a 
charter or non-public school in an intervening year, (2) dropped out of school prior to their 12 grade year, or (3) were otherwise missing 
course grades data in 10th or 11th grade. Because students who dropout prior to their 12% grade year are excluded from this sample, the 
graduation rates of those that continued are biased upwards. Unlike the data shown in the rest of this report, these analyses do not adjust 
for student or school characteristics, as the sample size is too small. 
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Table 1. First-time getting back on track: Grade-level when off-track ninth-grade students in 2013-14 and 2014-15 
cohorts first recovered, N=2,307 students continuously enrolled in SDP for four years 


COLUMNA COLUMN B 
... Who Graduated ...Who Did Not Graduate on 
Off-track 9'" Graders on Time Time 


% of Those % of Those 
% of All Who Got Who Got 
Off-Track Back on Back on Grad Rate 
# 9" Graders Track Track (unadjusted) 


Grade level when first got back on track 


262 


951 | az | 951 | 474m | | 
| nai2 | oom | 1350 | 95.6% | 62 | 44% 


All off-track students 


[as7 [om | - |---| - | 


Note: This analysis is restricted to off-track ninth-grade students for whom we had data on their on-track status for all years 
studied (43.1% of all off-track students). In addition, the definition of being On-Track in 12t grade is the same as the definition of 
what it takes to graduate. As a result, all students who got back on track in 12 grade graduated and none of the students who 
never got back on track graduated. The total sample size for this analysis was 2,307 students. 


Table 1 shows: 


e Most 9t graders who got back on track did so for the first time in 12‘ grade. Nearly 
70% of off-track 9t graders who got back on track did so in 12 grade (n=951), potentially 
reflecting historical district efforts targeting supports for seniors in order to get them to 
graduate. 

e While relatively few students were able to get back on track in 10“ or 11‘ grades, the 
vast majority who did graduated on time. About a third (18.6+14.7= 32.7%) of off-track 
9th graders got back on track for the first time in 10% or 11 grade. More than 85% of the 
students who got back on track in either of these grades graduated on time. 
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Student Characteristics Associated with Graduation Rates for 
Off-Track Students 


As discussed above, off-track 9t graders in the Classes of 2017 and 2018 had an on-time graduate 
rate of 57% - considerably lower than the 82% on-time graduation rates of their on-track peers. In 
order to understand if certain groups of off-track students might be more at risk of failing to 
graduate on-time, we examined disparities in recovery by student and school characteristics. As 
discussed in Box 3, the rates presented in this section are adjusted to account for factors that are 
jointly associated with ninth grade on-track status and graduation.12 


Off-track males had a graduation rate of 51%, nearly 12 percentage points 
lower than the grauation rate of 62% of off-track females. 


In the table below, we examine graduation rates of off-track students by subgroups defined by 
socio-demographic and other characteristics, including gender, racial/ethnic group, and low- 
income status (measured as Free from Tape status),13 special education status, and English Learner 
status. Looking only at students who were off-track following their first year of high school, we 
compare adjusted graduation rates across subgroups to understand any compounding risks factors 
for off-track 9th graders. 


Table 2. Four-year graduation rates for off-track first-time ninth-grade students in 2013-14 and 2014-15 cohorts, 
adjusted for school and student characteristics, by student subgroups 


62.3% 2,284 42.7% 


Race / Ethnicity 


White, Asian, another race, or multi- 53.2% 811 15.2% 
racial (ref 


Family low-income status in 9th grade (Free from Tape) 


Participated in social service programs 55.7% 3,694 69.1% 
Did not participate in social service 57.5% 168A 30.9% 
programs (ref 


Special education services in 9th grade 


12 For the technical reader, the statistical model specifies interactions between being off-track and student and school subgroup variables. 
See Appendix D for regression results. 


13 “Free from Tape” includes students who participate in SNAP, TANF, Medicaid, or other social service program 
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Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15, who did not permanently transfer out of SDP during the study period, and who 
were off-track following their first year of high school. The statistical model adjusts for school attended, school type and 
size, student gender, race/ethnicity, cohort, and 9th grade measures of average daily attendance, number of suspensions, 
and low-income, English Learner, and special education status. The total sample size for this analysis was 15,994 
students, but we are only reporting graduation rates here for off-track students (n=5,348). An * indicates the difference 
compared to the reference group in graduation rates was significant with a p-value of less than 0.05. 


Notably: 


e Among off-track students, females had moderately higher adjusted graduation rates 
than their male peers. Males were both more likely to be off-track and to fail to graduate 
than females.14 We found a moderate 12-percentage point difference between female and 
male students (p<0.05). 

e Among the off-track students, Black and Hispanic students had slightly higher on- 
time graduation rates than students of other race/ethnicities. Though more likely to be 
off-track to begin with,45 we observe a slight advantage of 6 percentage points between the 
graduation rate of black and Hispanic students and students of other race/ethnicities 
(p<0.05). 

e Off-track students enrolled in general education had slightly higher on-time 
graduation rates than their off-track peers receiving special education services. In 
addition to being much less likely to be off-track,16 off-track general education students 
overall were 7 percentage points more likely to graduate on time compared to their peers 
who were off-track and received special education services (p<0.05). 

e While slightly more likely to be off-track,!” there were not significant differences between 
the graduation rates for off-track students who did and did not received English Learner 
services in ninth grade or for students whose families were and were not low-income that 
year. 


14 See Table B-1 in Pileggi, M. & Strouf, K. (2018). On Track Across Four Cohorts: Ninth Grader On Track Patterns in the School District of 
Philadelphia, 2013-2017. The Philadelphia Education Research Consortium. https://www.phledresearch.org/on-track-4-cohorts 

15 Ibid 

16 Ibid 

17 Ibid 
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Suspensions and low attendance are important warning signals for off-track 
students. 


We also examined associations between on-time graduation and 9 grade suspensions and 
attendance (Figure 2). For suspensions, we compared the adjusted graduation rates for students 
with no suspensions, one suspension, and two or more suspensions in their 9th grade year. For 
attendance, we compared adjusted graduation rates at different levels, noting that a student is 
considered chronically absent if they attend less than 90 percent of the their 9t grade year: 90% or 
higher, 80-90%, and less than 80%. 


Figure 2. Four-year graduation rates for off-track first-time ninth-grade students in 2013-14 and 2014-15 cohorts, 
adjusted for school and student characteristics, by number of suspensions received and attendance rate in ninth 
grade 


100% 
90% 
80% 
3 
0, 
= 10% : . 62.4% 
fe) 57.9% 
Ss 60% 55.7% 
5 49.9% 49.8% 
ZS 50% x 
©) 
oa 40% 35.3% 
% 
2 30% 
ss} 
<x 
20% 
10% 
0% 
No suspensions One Two or more 90% orhigher 80%tolessthan Less than 80% 
(n=3,528) suspension suspensions (n=1,908) 90% (n=1,933) 
(ref) (n=238) (n=1,582) (ref) (n=1,507) 
Suspensions received in 9th grade Attendance rate in 9th grade 


Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15, who did not permanently transfer out of SDP during the study period, and who 
were off-track following their first year of high school. The statistical model adjusts for school attended, school type and 
size, student gender, race/ethnicity, cohort, and 9th grade measures of average daily attendance, number of suspensions, 
and low-income, English Learner, and special education status. The total sample size for this analysis was 15,994 
students, but we are only reporting graduation rates here for off-track students (n=5,348). An * indicates the difference in 
graduation rates was significant with a p-value of less than 0.05. 


This figure shows: 


e Both attendance and suspension are related to getting back on track and graduating 
on-time. 

o For off-track students, receiving at least one suspension in ninth grade isa 
clear warning sign. Off-track students who received one or more suspensions had 
an adjusted on-track graduation rate almost 10 percentage points lower than off- 
track students without any suspensions (p<0.05). 

o Attendance is an especially strong predictor, particularly for students with an 
average daily attendance below 80%. Off-track students with an average daily 
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attendance below 80% had an adjusted graduate rate of about 35%, or 27 
percentage points lower than their peers with 90% or higher average daily 
attendance (p<0.05). 


School Characteristics Associated with Graduation Rates for 
Off-Track Students 


In this section of the report, we assess whether off-track student graduation rates are associated 
with school characteristics, net of any increased risk of being off-track for some schools. These 
analyses draw attention to the challenge of supporting students at the school level. We examine the 
following characteristics of schools: 


e Admission type. High schools in SDP admit students in different ways.18 Neighborhood 
schools admit students based on catchment area, though some Neighborhood schools have 
specialized programs that draw students from around the city. Citywide schools admit 
students from the whole city, regardless of student residence address. Special Admission 
schools accept applicants based on criteria that may include grades, attendence, state test 
results, or disciplinary records. Prior PERC research, we found that neighborhood schools 
had the lowest on-track rates with values ranging from 49 to 61 percent compared to rates 
between 76 to 79% for Special Admit/Citywide schools, signaling that neighborhood 
schools are the ones with the greatest need for support. 

e School size. In addition to admission type, we examined differences by school size. 
Following research that shows school size is associated with student learning, we defined 
schools of 600 students or less as small schools, 601-900 students as medium schools, and 
schools with 901 or more students as large schools.19 


Off-Track students attending neighborhood schools are at significantly greater 
risk than those attending Citywide and Special Admission schools. 


Figure 3 shows differences in adjusted graduation rates for students based on the characteristics of 
their 9 grade school. The bars on the left show different adjusted graduation rates for students in 
Neighborhood, Citywide, and Special Admit schools. The bars on the right categorize schools by 
size, showing adjusted graduation rates for small, medium, and large schools. 


18 The School District of Philadelphia. (2019). High Schoo! Directory: Fall 2020 Admissions. 
https://www.philasd.org/studentplacement/wp-content/uploads/sites/19/2019/09/HS-Directory-2020.pdf 

19 Lee, V. E. and Smith, J. B. (1997). High school size: Which works best and for whom? Educational Evaluation and Policy Analysis, 19, 
205-227. 
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Figure 3. Four-year graduation rates for off-track first-time ninth-grade students in 2013-14 and 2014-15 cohorts, 
adjusted for school and student characteristics, by the type and size of high school they attended for ninth grade 


100% 
90% 
* * 

80% 
» 7 71.2% 
® 70% 68.3% 
=| 9 9 
60% 58.3% ae 58.4% 
= d 
=) 
S 50% 46.4% 
o) 
3B 40% 
a 
= 30% 
<x 

20% 

10% 

0% 
Neighborhood Citywide Special Admission Small Medium Large 
(n=3,706) (n=887) (n=755) (n=1,424) (n=1,225) (n=2,699) 
High school type High school size 


Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15, who did not permanently transfer out of SDP during the study period, and who 
were off-track following their first year of high school. The statistical model adjusts for school attended, school type and 
size, student gender, race/ethnicity, cohort, and 9th grade measures of average daily attendance, number of suspensions, 
and low-income, English Learner, and special education status. School size categories are defined as: Small = 600 or fewer 
students; Medium = 601-900 students; Large = 901 or more students. The total sample size for this analysis was 15,994 
students, but we are only reporting graduation rates here for off-track students (n=5,348). An * indicates the difference in 
graduation rates was significant with a p-value of less than 0.05. 


This figure shows: 


e The graduation rate for off-track students attending neighborhood schools in ninth 
grade was much lower than for off-track students at Citywide or Special Admission 
schools. The adjusted graduation rate at neighborhood schools trailed other schools by 
more than 20 percentage points (p<0.05). Increased risks for off-track students at 
neighborhood schools may reflect the fact that these students have more off-track students 
than other schools,”° straining recovery resources and pointing to the need for research on 
variable school capacity for supporting off-track students. 

e There was no association between school size and adjusted graduation rates. While 
graduation rates vary slightly by school size, these differences are not statistically 
significant. 


20 Crofton, M., & Neild, R. C. (2018). Getting On Track to Graduation: Ninth Graders’ Credit Accumulation in the School District of 
Philadelphia, 2014-2016. The Philadelphia Education Research Consortium. 
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Digging Deeper into the Off-Track Indicator 


With five distinct course requirements for being on-track at the end of ninth grade, students can be 
off-track for multiple reasons and to different degrees after their first year of high school. Here we 
look more deeply into how adjusted graduation rates vary depending on degree of off-track. To do 
this, we categorized 9 graders into a continuum of five groups based on the number of 
requirements to being on-track (one credit in each core course, plus one additional credit, as 
explained in Box 1) and the total number of credits they earned.2! Table 3 below provides 
definitions of these categories, as well as the number and percentage of students falling into each 
category. 


Table 3. Definition of categories of on- and off-track students and the number and percentage of first-time ninth- 
grade students in 2013-14 and 2014-15 cohorts in each category 


Securely On-track students who earned more than 6 
On-Track credits total. 


On-track students who earned exactly one 
credit in each core course and 5-6 credits 
total. 


Marginally 
On-Track 


Off-track students missing a single 
requirement. These students had not earned 
one of the core subject credits or, if they had 
earned all of those credits, had not 
completed at least one additional credit. 


Marginally 
Off-Track 


Moderately Off-track students that did not meet two or 
Off-Track three of the five requirements. 


Off-track students missing four or five of the 


Far Off Ttack requirements for being on track. 


Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15, who did not permanently transfer out of SDP during the study period, and who 
were off-track following their first year of high school. The total sample size for this analysis was 15,994 students, but we 
are only reporting graduation rates here for off-track students (n=5,348). 


21 These off-track categories are the same three categories used for the prior PERC report Getting On Track to Graduation, though we 
renamed the first category to “Marginally Off-Track” from “Almost On-Track” for clarity in this report.The on-track categories presented here 
differ from the on-track categories used in SDP’s 2020 report on college matriculation of on-track ninth graders (Tanz., A. & Erdem-Akcay, 
E. (2020). From Ninth Grade On-Track to College Matriculation: The Path of the 2015-16 SDP Ninth-Grade Cohort. The School District of 
Philadelphia. https://www.philasd.org/research/2020/06/04/from-ninth-grade-on-track-to-college-matriculation-the-path-of-the-2015-16- 
sdp-ninth-grade-cohort/). That report uses the grades earned in courses as the differentiaion between a student being “Firmly On-Track” 
versus “On-Track But At Risk”, whereas we use the number of total credits a student earned. 
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The farther off track a student was at the end of 9! grade, the lower their 
adjusted graduation rate. 


Using these definitions for degree of on- and off-track, we modeled graduation rates for all students 
based on their on-track status at the end of ninth grade (Figure 4). 


Figure 4. Four-year graduation rates for first-time ninth-grade students in 2013-14 and 2014-15 cohorts, adjusted for 
school and student characteristics, by degree on- or off- track in ninth grade 


MISSING 0 REQUIREMENTS 
AND EARNING MORE THAN 6 CREDITS TOTAL 


MISSING 0 REQUIREMENTS, 5 
BUT EARNING ONLY 5-6 CREDITS TOTAL 82.2% 
GRADUATION RATE 
OF ALL ON TRACK STUDENTS 


68.5% 


MISSING 1 
REQUIREMENT 


49.8% 


MISSING 2-3 
REQUIREMENTS 


LY) 
26.9% 56.5% 


MISSING 4-5 GRADUATION RATE 
REQUIREMENTS OF ALL OFF TRACK STUDENTS 


>. ep. -_p- ap. >» 


Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15, who did not permanently transfer out of SDP during the study period, and who 
were off-track following their first year of high school. The statistical model adjusts for school attended, school type and 
size, student gender, race/ethnicity, cohort, and 9th grade measures of average daily attendance, number of suspensions, 
and low-income, English Learner, and special education status. The total sample size for this analysis was 15,994 
students. 


Notably: 


e The farther off-track students were during their ninth-grade year, the lower their 
graduation rates. The more than 1,100 far off-track students had by far the lowest 
graduation rates, with about one quarter graduating on time. Because this category 
captures students who essentially began 10 grade a full year behind their peers, this is not 
surprising, but the scale of the challenge is clear from the difference in these percentages. 


15 | Getting Back On Track 


e Even for students on the cusp of being on- or off-track, being on track at the end of 
ninth grade meant they were substantially more likely to graduate on time. Marginally 
on-track students had an adjusted graduation rate 9 percentage points higher than 
marginally off-track students. 


For students missing only one credit, it mattered little which credit requirement 
was missing. 


As defined in Box 1, four of the five requirements to be on track at the end of ninth grade are tied to 
earning at least one credit in specific subject areas: math, English, science, and social studies. The 
fifth requirement is to earn one additional credit in any subject area. If any one of these five 
requirements were more consequential for eventual graduation, specific attention could be paid to 
student progress in that subject area in ninth grade. 


To assess the relative importance of these five requirements, we examine the adjusted graduation 
rates based on the subject area of the missing credit for marginally off-track students (who were 
missing only one credit). We restricted this analysis to only those students who were marginally 
off-track (42.3% of off-track students), who had an adjusted graduation rate of 68.5% (Figure 4, 
above). As Table 4 shows, differences across categories were small.22 It is notable that students 
missing the math credit graduated at 10 percentage points lower than those missing social studies - 
a finding that suggests the need for further examination. Yet, overall this difference is moderate and 
not statistically significant. 


Table 4. Four-year graduation rates for marginally off-track first-time ninth-grade students in 2013-14 and 
2014-15 cohorts, adjusted for school and student characteristics, by the credit they were missing in ninth 
grade 


Social Studies 73.5% 49] 9.2% 4 
English 68.6% 383 7.2% 4 
Science 68.4% 662 12.4% 3 
Math 63.5% 694 13.0% 3 


Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15, who did not permanently transfer out of SDP during the study period, and who 
were off-track following their first year of high school. The statistical model adjusts for school attended, school type and 
size, student gender, race/ethnicity, cohort, and 9th grade measures of average daily attendance, number of suspensions, 
and low-income, English Learner, and special education status. The total sample size for this analysis was 15,994 
students, but we are only reporting graduation rates here for marginally off-track students who were missing a core 
course credit (n=2,230), excluding 34 marginally-off track students missing a non-core course credit. 


22 Since very few marginally off-track students were missing the fifth requirement of an “additional credit” (only 34 students), we excluded 
them from this analysis. 
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The Use of Credit Recovery to Get Back On Track 


Knowing the graduation rates for different groups of students helps us understand which students 
might be in need of more support through their high school years. But how can schools provide that 
support? As discussed earlier, most students who got back on track did so in their senior year. This 
next section examines students who were able to get back on track in their senior year to try to 
understand if credit recovery opportunities in that year played a role. 


How do students get off track? There are two ways a student can end their 9 grade year off-track. 
An off-track student might have failed one or more required courses in their freshman year, or they 
might not have enrolled in one or more courses bearing a required credit. 


When does a student need to recover a credit from 9 grade? Not all students who fail a course 
in 9th grade need to recover that course credit in order to graduate. Even if the course is in a core 
subject, policies for which core courses are prerequisites for others vary by school. So, the need to 
recover the credit for graduation depends on whether students need it to progress in the subject 
area and gain the credits required for graduation. 


What are student options for recovering a missing course? A student has three options to earn a 
missing course credit. For students who were not enrolled in the first place, they must enroll in the 
full course in a future year. If a student took the course but failed, they can recover the credit either 
by retaking the course completely or by enrolling in a credit recovery. 


What are the different alternative credit recovery formats? There are two credit recovery 
formats: (1) face-to-face classroom-based recovery and (2) web-based recovery.23 These two 
recovery formats differ from retaking the course in full because the courses are taken on a shorter 
timeline and with a modified structure compared to retaking the course in a traditional format. For 
this analysis, we focus on alternative credit recovery formats. 


Note on data availability. Data on whether a failed course was retaken in a traditional format or an 
alternative credit recovery format was first captured in SDP’s data system for the 2017-18 school 
year. Students may have attempted or regained credits using alternative credit recovery formats 
before the 2017-18 year, but we could not identify those cases in the data. For that reason, this 
analysis is restricted to the 2017-18 school year and uses only off-track 9t» graders in the Class of 
2018 (Cohort 2) and their course recovery pathways during their senior year year.24 That means 
our analysis in this section is restricted to just 30% of the sample of off-track students that we have 
been studying through the rest of the report (62% of the off-track sample of Cohort 2 students). We 
present unadjusted graduation rates because the sample size is too small to support a model-based 
approach. 


23 The options available to a student vary by school, however, as schools have the flexibility to determine which formats of courses are 
offered to their students. 

24 We elected to leave Class of 2017 students out of this analysis because the number of students who remained in the district for a fifth 
year was much smaller than the overall cohort off-track students and they likely look substantially different than other students in that 
cohort. Those differences may have impacted whether or not they took credit recovery courses thus we did not include them in this 
analysis. 
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While representing only a fifth of off-track students, most who used Credit 
Recovery in 2017-18 to recover credits graduated. 


Figure 5 shows how many off-track ninth graders in the Class of 2018 (Cohort 2) attempted and 
recovered credits during their senior year by each mode of alternative credit recovery and the 
unadjusted graduation rates of each group. In this figure, four of five students did not use credit 
recovery (81.7%). The remaining, while only representing a little less than a fifth of off-track 
students (18.3%), at least attempted Credit Recovery in 2017-18 to recover credits.26 


Figure 5. Number, percent, and unadjusted graduation rates of off-track ninth-grade students in 2014-15 
cohort, by whether or not they used credit recovery in 2017-18, n=1,600 


Recovered any credits via Credit Recovery courses 


OFF TRACK STUDENTS 225 STUDENTS 
N=1600 14.1% OF COHORT 2 
Attempted any Credit Recovery courses 83.6% 
293 STUDENTS 
18.3% OF COHORT 2 
66.9% 


Did not use credit recovery 
1307 STUDENTS 
81.7% OF COHORT 2 


Note: Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2014-15, who did not permanently transfer out of SDP during the study period, who were off-track 
following their first year of high school, and who had course enrollment data for the 2017-18 school year. The total 
sample size for this analysis was 1,600 students. 


Notably: 


e Most of the students who attempted credits via credit recovery were able to regain 
credits. Out of 293 students who attempted a credit recovery course, three-fourths 
(N=225) were able to recover those credits. 

e Students who recovered credits via Credit Recovery courses had much higher 
graduation rates than their peers who did not attempt credit recovery through such 
courses. Most students who recovered credits graduated (83.6%) compared to about half 


25 Appendix Table C7 shows the graduation rates and number of students who attempted and who recovered credits by mode of 
alternative credit recovery (classroom face-to-face or web-based). Students may have taken recovery courses using both formats, in which 
case they are included in both rows of that table. 


26 Appendix Table B2 shows the percent of students who attempted Credit Recovery by degree of off-track. 
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of those who did not attempt to use Credit Recovery (55.2%), a difference of about 28 
percentage points (p<0.05). 

e Students who attempted credits, even if they did not earn those credits, had higher 
graduation rates than those that did not. Two-thirds of those that attempted to recover a 
course through credit recovery graudated on time (66.9%), compared to about half (55.2%) 
of those who did not use credit recovery, a difference of about 12 percentage points 
(p<0.05). 


Implications for policy and practice 


Continuing to focus on getting students on track after their first year of high 
school is an important strategy for improving the high school graduation rate. 


Ninth grade is a critically important year, and how a student manages the transition to high school 
is a strong indicator of their likelihood of graduating three years later. This research builds on prior 
research on SDP’s Ninth Grade On-Track indicator and continues to make the case that students on 
track at the end of ninth grade are more likely to graduate on time or within 5-years of starting high 
school. A strong dropout prevention strategy should involve careful review of student-level data to 
identify 9 graders at risk of dropping out. With the Ninth Grade On-Track indicator, SDP high 
school school staff have access to a concise indicator for flagging students at risk early on. 


The district should consider expanding the tracking system to include degree of 
off-track, as well as support school staff in reviewing the On-Track indicator 
alongside other student-level data. 


Though there is parsimony and simplicity in a dichotomous indicator for flagging off-track students, 
our analysis above shows that recovery rates to eventual graduation are lower for some students. 
Reviewing a broader range of student level data in concert with the On-Track indicator will give 
schools a more nuanced understanding of the level of risk for their students. For example, far- and 
moderately off-track students have graduation rates far below their marginally off-track peers. 
Additionally, male students, students receiving Special Education services, and Black and Hispanic 
students show increased risk of failing to recover from being off-track in 9 grade. For other off- 
track students, paying attention to suspensions or attendance challenges might be used to flag 
students for more comprehensive and continuous supports both before and udring attempts to 
retake courses to get them back on track. 


All off-track students need support, and helping off-track students recover may 
require different solutions for different students and in different school settings. 


While this study did not examine school variation in approaches to getting students back on track, 
the evidence of varying levels of recovery by student and school characteristics provides fodder for 
a discussion of what works to support students. We consider support for all students to be a crucial 
strategy for improving graduation rates. Focusing, for example, only on far or moderately off-track 
students would have left over 2,200 marginally off-track students unsupported, among whom, as 
our analysis shows over 700 failed to recover. Thus approaches should be designed as inclusive of 
all off-track students. 
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However, school supports for off-track students should match the reasons for course failure in 9% 
grade as well as reasons students struggle to recover. Thus, crucial next steps for the district 
include understaing why students fail, what works to get students back on track, and the 
relationship between the two. While not definitive based on the data presented here, research in 
other districts suggests that off-track 9th graders usually become disengaged from school in 
primarily one of two ways: one rooted in an academic struggle and another in misbehavior or 
aversion to school attendence.27 


While it may be unduly burdensome for the district to regularly provide school administrators with 
the kind of extensive and nuanced student-level data on root causes for academic failure and failure 
to recover, a study of the correlation between reasons for disengagement and recovery and degree 
of off-track and other student and school characteristics would be a reasonable next step. Armed 
with those corrleations, the district could leverage readily available student level data that is 
strongly associated with root causes and likelihood of recovery to enable schools to move toward 
matching students to effective supports. 


This study also shows that schools matter and suggests that schools with many off-track students 
are suffering from an untenable strain on resources for off-track students’ recovery. The district 
should consider more research on school variation in capacity to support off-track students, 
focused on strategies used across school admission types for preventing course failures and for 
supporting the recovery of off-track students, and what works in which settings. 


27 Balfanz, R., Herzog, L., & Mac Iver, D. M. (2007). Preventing student disengagement and keeping students on the graduation path in 
urban middle-grades schools: Early identification and effective interventions. Educational Psychologist, 42, 223-235. 
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Appendix A: Student and School Variable Definitions 


Key variables for this analysis were defined in Box 2 of this report. Further detail about the 
variables used to classify students and schools are provided below. 


Race / Ethnicity: Categorical variable for if the student identified as Black, Hispanic, or 
another race (including: American Indian or Alaskan Native, Asian, Native Hawaiian or 
Pacific Islander, White, or multiracial) 


Gender: Indicator of whether the student identified as male or female. 


Low-income status in 9th grade: Indicator of whether a student’s family received public 
assistance through Temporary Assistance for Needy Families (TANF), Supplemental 
Nutrition Assistance Program (SNAP), Medicaid, or other social service programs. While 
this measure identifies the most economically disadvantaged students, it underestimates 
the number of students living at or close to poverty compared to the previously available 
Free and Reduced Price Lunch eligibility measures. 


Special education status in 9th grade: Indicator of whether students were receiving 
special education services other than gifted and talented services. 


English learner status in 9th grade: Indicator of whether a student was classified as 
English learner and receiving linguistic support. 


Courses taken in 2017-18: All courses the student was enrolled in for the 2017-18 school 
year and the mode the course was taken in (e.g. in-person or web-based credit recovery). 


High school attended: The school each student attended for ninth grade. For students who 
attended more than one school in their ninth grade year, we used the last school attended. 


High school type: Based on SDP classifications of high schools, each high school was 
identified as a Neighborhood school (which primarily draws students from a catchment 
area based on the elementary school the student attended), a Citywide school (which draws 
students from all over Philadelphia and has a lottery-based admission process), or a Special 
Admission school (which are “magnet” schools with specific admissions requirements). 


High school size: Based on the total number of students enrolled in the school in the 2013- 
14 school year (or the first year it operated for schools that opened in 2014-15), the 
research team developed an indicator for small (600 or fewer students), medium (601-900 
students), or large (901 or more students) schools. 
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Appendix B: Sample Size by Degree of Off Track, 
Missingness and Credit Recovery Attempts 


Table B1. Number and percent of all ninth-grade students with valid and missing data in 2013-14 and 2014-15 
cohorts, by on-/off-track status 


On- / Off- Track Status in 9% Number of Students RUIaBen Ot Students Percent of Students 


with Missing Data 


2 with Missing Data in 
Grade in Both Cohorts Both Cohorts 
Securely On-Track 

(earning all 5 requirements and 

more than 6 credits total) 

Marginally On-Track 

(earning all 5 requirements, but 

only 5-6 credits total) 


Marginally Off-Track 


(Missing 1 requirement) 


Moderately Off-Track 
(Missing 2-3 requirements) 


Far Off-Track 
(Missing 4-5 requirements) 


All 9th Grade Students 


Table B2. Number and percent of off-track ninth-grade students in 2014-15 cohort, by whether or not they 
attempted any credit recovery in 2017-18 
Number of Off-Track | Number of Off-Track | Percent of Off-Track 
9" Grade Students 9" Grade Students 9" Grade Students 
Off- Track Status in 9" Grade Who Did Not Who Attempted Any Who Attempted Any 
Attempted Credit Credit Recovery Credit Recovery 
Recovery Courses Courses Courses 


Marginally Off-Track 
(Missing 1 requirement) 


20% 


i ‘i 629 
Moderately Off-Track 6 
(Missing 2-3 requirements) 

Far Off-Track 


All Off-Track Students vie} 
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Appendix C: 4- and 5-year Adjusted and Unadjusted 
Graduation Rate Tables 


Table C1. Four- and five-year adjusted and unadjusted graduation rates for first-time ninth-grade students in 2013- 
14 and 2014-15 cohorts, by their ninth grade on-track status, N=15,994 


9th Grade Model-Adjusted Unadjusted Percentage-Point 
On-Track Status Graduation Rates Graduation Rates Difference 


eens Off-track 56.5% 43.9% 
graduation rate 
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Table C2. Four-year adjusted and unadjusted graduation rates for first-time ninth-grade students in 2013-14 and 
2014-15 cohorts, by student characteristics and cohort, N=15,994 


Number of Students 


Adjusted On-Time Unadjusted On-Time 
Graduation Rates Graduation Rates 


On-Track  Off-Track9th | On-Track Off-Track | On-Track9th | Off-Track 
9th Graders (eTe-le(-163 9th Graders | 9th Graders (eTr1e (163 9th Graders 


79.1% 50.7% 84.5% 38.5% 4,886 3,064 
85.6% 62.3% 91.0% 51.1% 5,760 2,284 


Special education services in 9th grade 


Received 

Did not 

receive 81.9% 57.6% 89.0% 47 3% 9,405 4,016 
services 


Race / Ethnicity 


81.5% 58.7% 87.7% 45.7% 5384 3,393 
79.8% 59.2% 82.8% 40.8% 1,178 1,144 


White, 

Asian, or 

another 83.7% 53.2% 91.3% 40.6% 3,484 811 
race 


Family low-income status in 9th grade (Free from Tape) 


Participated 

nsoetal 81.9% 55.7% 85.1% 40.7% 5,495 3,694 
service 

programs 

Did not 

participate 

in social 82.9% 57.5% 91.1% 51.1% 5,151 1,654 
service 

programs 


English learner services in 9th grade 


Received 

Did not 

receive 82.2% 56.5% 88.5% 44.0% 9,621 4,804 
services 


Suspensions received in 9th grade 


84.9% 57.9% 89.8% 49.9% 9,718 3,528 
suspensions 


2 or more 
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Adjusted On-Time Unadjusted On-Time 
Graduation Rates Graduation Rates 


On-Track Off-Track 9th On-Track Off-Track || On-Track 9th Off-Track 
9th Graders Graders 9th Graders | 9th Graders Graders 9th Graders 


Attendance rate in 9th grade 


90% or 
ene 90.4% 62.4% 93.2% 60.3% 1,908 
80% to less 
— 64.8% 35.3% 59.0% 24.4% 705 1,933 


Cohort 


2013-14 83.3% 56.4% 88.6% 44.0% 5,305 2,779 


Number of Students 
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Table C3. Four-year adjusted and unadjusted graduation rates for first-time ninth-grade students in 2013-14 and 
2014-15 cohorts, by type and size of school attended in ninth grade, N=15,994 


Adjusted On-Time Unadjusted On-Time 
Graduation Rates Graduation Rates 


On-Track 9th Off-Track On-Track 9th Off-Track On-Track Off-Track 
Graders 9th Graders Graders 9th Graders | 9thGraders | 9th Graders 


High school admission type 


Neighborhood 78.3% 46.4% 79.5% 34.3% 4,560 3,706 
Ciywide asx | on | vox a 


special 90.7% 71.2% 95.6% 73.2% 4804 755 
Admission 


High school size 


Small 

fe id de i 
students 

Medium 

(601-900 80.6% 54.0% 89.7% 47 8% 2,886 1,225 
students 

Large 

(901 or more 83.2% 58.4% 86.6% 37.7% 4,937 2,699 
students) 


Number of Students 
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Table C4. Four-and five-year adjusted and unadjusted graduation rates for on- and off-track first-time ninth-grade 
students in 2013-14 and 2014-15 cohorts, by their degree of off-track in ninth grade, N=15,994 


Adjusted Graduation Rates Unadjusted Graduation Rates Number of 
On- / Off- Track Status in 9" Grade Students in 
This Group 


4-Years | 5-Years 4-Years 5-Years 


Securely On-Track 

(earning all 5 requirements and 83.5% 85.8% 88.8% 90.5% 9,008 
more than 6 credits total) 

Marginally On-Track 

(earning all 5 requirements, but only 77.7% 81.0% 83.9% 86.8% 1,638 
5-6 credits total) 


Marginally Off-Track 

(Missing 1 requirement) 68.5% 73.6% 63.8% 69.6% 2,264 
Moderately Off-Track 

(Missing 2-3 requirements) eee pene Sie AGd 9c 
Far Off-Track 

(Missing 4-5 requirements) 26.9% 38.0% 14.6% 22.3% 1,124 
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Table C5. Four-year adjusted and unadjusted graduation rates for off-track first-time ninth graders in 2013-14 and 
2014-15 cohorts, by the requirement(s) they were missing in ninth-grade, N=15,994 


If the Student Total Credits 
Was Missing Adjusted UTE Ye JIVE cto] Number of Off- Percent of Off- Needed in 
the Credit Graduation Rates | Graduation Rates — Track Students Track Students Subject Area to 
Graduate 


All Off-Track 
Students 


100.0% 
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Table C6. Which credit off-track first-time ninth-grade students in 2013-14 and 2014-15 cohorts were missing in 


ninth grade, N=5,348 


Missing Requirement 


Marginally Off-track Students (missing only one requirement) 
Only missing math 

Only missing science 

Only missing social studies 

Only missing English 


Only missing the additional credit 


Number of Off-Track Percent of Off-Track 
Students Students 


Moderately and Far Off-track Students (missing more than one requirement, so each student included in multiple rows) 


Missing math plus at least one other requirement 
most commonly science 

Missing science plus at least one other requirement 
most commonly math 

Missing social studies plus at least one other 

requirement (most commonly science 

Missing English plus at least one other requirement 
most commonly math 

Missing the additional credit plus at least one other 

requirement (most commonly English 
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Table C7. Number, percent, and unadjusted graduation rates of off-track ninth-grade students in 2014-15 cohort, by 
whether or not they used each mode of credit recovery in 2017-18, N=1,600 


Course Recovery Enrollment in 2017- Number of Off-Track | Percent of Off-Track 9* Unadjusted 
18 9" Grade Students Grade Students Graduation Rates 


a not use credit recovery in 2017- 1,307 81.7% 55.2% 
Attempted any Credit Recovery 293 18.3% 66.9% 
courses 

Recovery 


ea any credits via Credit 225 14.1% 83.6% 
ecovery courses 
Recovery 


Any Web-based Recovery 80.6% 


Any Web-based Recovery 63.0% 


32 | Getting Back On Track 


Appendix D: Model Building Process and Model Regression 
Results 


As discussed in Box 3, we employed a multi-level regression model to estimate marginal graduation 
rates for the study sample rather than presenting unadjusted rates. Marginal or “adjusted” 
graduation rates account for factors that are jointly associated with ninth grade on-track status and 
graduation. Factors that might influence both “on-track status” as well as graduation from high 
school, include student demographic characteristics, ninth grade program participation (e.g., 
reciept of Special Education and English Learner services), school attendance rates and behavior 
incidents in ninth grade, and the school the student attended in ninth grade. 


The final multi-level statistical model specificies students nested within schools where the student 
finished their freshman year. To identify the covariates and interaction terms that provided the 
best model fit, researchers engaged in a step-wise model building process, comparing log- 
likelihoods with the addition of covariates (Table D1). Table D2 shows regression coefficients and 
standard errors of all variables used in models 1, 7, and the final model. 


Table D1. Model fit indices used during model-building to assess addition of covariates 
Model 1 Empty model predicting on-time -8051.81 
graduation 


Model 2 Add off-track status binary indicator -6986.63 2130.37 


Model 3 Add student-level demographic -6920.81 
characteristics 


Model 4 Add student-level characteristics -6897.74 
(special education, English Learner, 
and low-income) 
Model 5 Add 9th grade attendance -6579.68 
Model 6 Add 9th grade suspensions -6533.55 


Model 7 Add school size and school type -6506.46 
variables 


Add interaction terms -6455.58 
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Table D2. Logistic regression coefficients for on time graduation regressed on student and school characteristics, 
N=15,994, School District of Philadelphia, Classes of 2017 and 2018 


Model 7 - Main 


Model prior to Pina Modet 
Model 2 - : with significant 
Base Model ; aggIne interaction 
interaction 
pane terms 
Off-Track in 9t Grade -1.978*** -1.555*** -1.570*** 
(0.0449) (0.0477) (0.147) 
Student-level covariates 
Female 0.569*** 0.567*** 
(0.0459) (0.0459) 
Black 0.0218 -0.128 
(0.0654) (0.0856) 
Hispanic -0.0806 -0.260*** 
(0.0742) (0.0994) 
Cohort 1 student (first enrolled in SY2013-2014) 0.0787* 0.190*** 
(0.0453) (0.0652) 
Received English Learner services in 9b grade -0.00696 -0.00476 
(0.0767) (0.0772) 
Low-income status in 9th grade -0.0994** -0.0887* 
(0.0487) (0.0490) 
Receieved special education services in 9th grade -0.195*** 0.104 
(0.0556) (0.0894) 
9th grade Average Daily Attendance (ADA) between 
80% and 90% 0.842*** 0.808*** 
(0.0620) (0.0987) 
9th grade ADA >= 90% 1.487*** 1.802*** 
(0.0617) (0.0963) 
Student received 1 Suspension in 9 grade -0.541*** -0.800*** 
(0.117) (0.174) 
Student received 2 or more suspensions in 9» 
grade -0.510*** -0.844*** 
(0.0597) (0.0978) 
School-level covariates 
Attended a Special Admission school for 9t* grade 1.132*** 1.075*** 
(0.123) (0.123) 
Attended a Citywide school for 9th grade 0.749*** 0.728*** 
(0.139) (0.138) 
Attended a school for 9» grade that enrolled fewer 
than 600 students 0.128 0.125 
(0.117) (0.117) 
Attended a school for 9b grade that enrolled more 
than 900 students 0.219 0.190 
(0.135) (0.134) 
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Model 7 - Main 


Model prior to Era Mode : 
Model 2 - . with significant 
Base Model . AgGINE interaction 
interaction 
eecrae terms 
Interaction Terms 
Offtrack_g9#Black 0.371*** 
(0.120) 
Offtrack_g9#Hispanic 0.426*** 
(0.140) 
Offtrack_g9#Cohort1 -0.202** 
(0.091) 
Offtrack_g9# Special education status -0.446*** 
(0.115) 
Offtrack_g9# ADA between 80% and 90% 0.138 
(0.125) 
Offtrack_g9# ADA >= 90% -0.542*** 
(0.121) 
Offtrack_g9#1 Suspension 0.414* 
(0.229) 
Offtrack_g9#2 or More Suspensions 0.453*** 
(0.119) 
L2 - Last School Enrolled in 9 grade 0.819*** 0.0797*** 0.0779*** 
0.178 0.0264 0.0260 
Constant 2.167*** 0.0871 0.0185 
0.131 0.141 0.159 
Observations 15,994 15,994 15,994 
Number of groups 52 52 52 
chi2 1943 2716 2724 


Standard errors in parentheses 
** 4 <0.01, ** p<0.05, * p<0.1 


Note. Author calculations using data for School District of Philadelphia traditional high school students who first enrolled 
in ninth-grade in 2013-14 or 2014-15 and who did not permanently transfer out of SDP during the study period. 
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